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SPATIO-TEMPORAL FILTER UNIT AND IMAGE DISPLAY APPARATUS 
COMPRISING SUCH A SPATIO-TEMPORAL FILTER UNIT 

BACKGROUND OF THE INVENTION 
Field Of The Invention 

[0001] The invention relates to a spatio-temporal filter unit, 
provided with a sigma filter, to reduce the noise in images. 
5 [0002] The invention further relates to an image display 
apparatus comprising such a spatio-temporal filter unit. 

Description Of The Related Art 

[0003] A unit of the kind described in the opening paragraph is 
10 known from G. de Haan et al, "Memory integrated noise reduction for 
television," in IEEE Transactions on Consumer Electronics, Vol. 42, 
No 2, May 1996, Pages 175-181. In this article, a unit is described 
which is a cascaded system of two filters, i.e., a spatial filter 
and a temporal filter. The spatial and temporal filters together 
15 should result in effective noise reduction. In general, spatial 
filters require pixels from one image, while temporal filters 
require samples from two or more successive images. 
[0004] The spatial filter is a recursive sigma filter. It is 
based on a variant of the sigma nearest neighbors selection 
20 process. The combination process involves weighted averaging, which 
is optimal for additive white Gaussian noise. The weights of the 
described sigma filter can have three possible values, which are 
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selected based on thresholds that are adapted to the noise level. 
In the article, also a noise estimator is described to estimate the 
noise level. 

[0005] The temporal filter is realized as a motion- compensated 
5 first -order temporal recursive filter. Therefore, this temporal 

filter requires a motion estimation and motion compensation system, 
which makes the concept expensive. Furthermore, a motion detection 
means is necessary to adapt the strength of the filtering when 
motion occurs. Otherwise, the image is distorted. 

10 

SUMMARY OF THE INVENTION 
[0006] It is a first object of the invention to provide a 
spatio-temporal filter unit, of the kind described in the opening 
paragraph, that integrates spatial and implicit mot ion -compensated 

15 temporal noise reduction in one filter. For the motion 

compensation, no motion vectors, or other means to express the 
amount and direction of movement, are required. 
[0007] It is a second object of the invention to provide an 
image display apparatus comprising such a spatio-temporal filter 

20 unit. 

[0008] The first object of the invention is achieved in that the 
spatio-temporal filter unit comprises one filter kernel designed to 
operate on pixels from both a current image and from the output of 
the spatio-temporal filter unit, being a temporally recursive 
25 filtered image. The filtering is based on the sigma nearest 
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neighbors selection. Pixels that have poor correlation with the 
original pixel are automatically discarded, whereas other pixels 
contribute to the output in proportion to their correlation. It 
involves weighted averaging of selected pixels from a predefined 3- 
5 dimensional window comprising horizontal, vertical and temporal 
directions. A major characteristic of this spatio-temporal filter 
unit is that the spatial and temporal filtering are done within the 
same filter kernel. This reduces the implementation cost 
significantly and leads to an easily optimized concept. Moreover, 

10 problems, such as the so-called "dirty- window" effect or frozen 
noise, "comet tails," etc., are automatically solved since the 
temporal filter is no longer separable. Noise breakthrough at edges 
or around moving objects is also eliminated because the spatial and 
temporal filters complement each other. 

15 [0009] An embodiment of the spatio-temporal filter unit 
according to the invention comprises: 

- a spatial pixel buffer for storing pixels of the current image; 

- a spatial pixel selector for selecting pixels from the spatial 
pixel buffer; 

20 - a temporal pixel buffer for buffering pixels from the output of 
the spatio-temporal filter unit, said output being a temporally 
recursive filtered image; 

- a temporal pixel selector for selecting pixels from the temporal 
pixel buffer; and 
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- the sigma filter comprising one filter kernel designed to operate 
on the pixels from both the spatial pixel selector and the temporal 
pixel selector. 

[0010] In the spatial pixel buffer, a 2 -dimensional spatial 
5 pixel selector is defined to select pixels to enable spatial 

filtering. In order to add temporally recursive filtered pixels in 
the filter kernel of the sigma filter, a second 2 -dimensional 
temporal pixel selector is defined in the temporal pixel buffer, 
which is filled with pixels taken from a picture memory. In this 

10 way, 3 -dimensional spatio-temporal filtering is realized within one 
filter aperture. The output of the sigma filter is stored in a 
picture memory and used for subsequent filtering. This is the 
temporal recursive part of the processing. Motion detection is 
inherent in the filtering. Each pixel is first compared with the 

15 original one to determine its contribution to the output. Where 
motion occurs, the pixel will have a large difference in gray 
value. Therefore, it will be automatically discarded. Furthermore, 
motion compensation is also inherent. This is realized by using a 
temporal pixel selector for the selection of the temporally 

20 recursive filtered pixels and a discriminating averaging filter, 
which uses a threshold. If there is no motion, the gray level 
correlation relative to the central input pixel is high around the 
central pixel of the previous image. When motion occurs, this 
correlation is shifted in position automatically following the 

25 motion trajectory. Provided the motion is low enough, temporal 
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filtering will take place because correlated pixels can be found 
somewhere within the temporal window, i.e., the temporal pixel 
selector. In other words, motion compensation is implicitly done, 
thereby making explicit motion compensation redundant. 
5 [0011] An embodiment of the spatio-temporal filter unit 

according to the invention comprises an adaptive sigma filter. The 
filter thresholds of the sigma filter are adjustable. That means 
the degree of filtering can be increased or decreased to suit the 
taste of the user of this embodiment of the spatio-temporal filter 

10 unit, or of an image display apparatus comprising such a spatio- 
temporal filter unit. The thresholds for spatial and temporal 
filtering can be independently determined so that the desired 
balance between spatial and temporal filtering is achieved. 
[0012] An embodiment of the spatio-temporal filter unit 

15 according to the invention is characterized in that of at least one 
of the pixel selectors, being the temporal pixel selector and the 
spatial pixel selector, the aperture can be adjusted. This 
determines the strength of the filtering and the characteristics of 
the output. A combination of a narrow aperture and a large aperture 

20 with image alternation gives good results. 

[0013] An embodiment of the spatio-temporal filter unit 
according to the invention is characterized in that at least one of 
the pixel selectors, being the temporal pixel selector and the 
spatial pixel selector, is designed such that the distance between 

25 the selected pixels is adjustable. The result is that the aperture 
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of the selector can be adjusted while keeping the number of 
selected pixels more or less constant. The costs of the pixel 
selectors are based on their storage capacity, i.e., the number of 
pixels they can store simultaneously. The advantage of this 
5 embodiment according to the invention is that the costs for 
processing of the pixels can be relatively low. 
[0014] An embodiment of the spatio-temporal filter unit 
according to the invention comprises a motion detector designed to 
detect motion and to control the aperture of the temporal pixel 

10 selector based on detected motion. The aperture determines the 
strength of the filtering for still images and the range of the 
motion compensation for moving sequences . An advantage of such an 
embodiment is that the implicit motion compensation is optimized. 
The motion detector may detect global or local motion. As the 

15 aperture can be determined on a field basis, an optimal setting can 
be found for each sequence once the degree of motion activity is 
known. The detector does not have to provide information about the 
direction of movement. It is also possible that an external motion 
estimator provides information about the estimated motion. 

20 [0015] An embodiment of the spatio-temporal filter unit 

according to the invention comprises a motion estimator designed to 
supply motion vectors and to control the position of the temporal 
pixel selector related to the temporal pixel buffer, based on the 
motion vectors supplied by the motion estimator. The advantage of 

25 such an embodiment is that the temporal proximity is increased. 
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This is especially beneficial in the case of high object 
velocities. It is also possible that an external motion estimator 
provides the motion vectors. 

[0016] An embodiment of the spatio-temporal filter unit 
5 according to the invention comprises a noise estimator designed to 
estimate a noise level in the current image and to control the 
aperture of at least one of the pixel selectors, being the temporal 
pixel selector and the spatial pixel selector, based on the 
estimated noise level . As noted previously, the strength of 

10 filtering is related to the aperture of the pixel selectors. 

Relatively more filtering is allowed in case of a relatively high 
estimated noise level. Therefore, optimal noise filtering is 
achieved at all noise levels. It is also possible that an external 
noise estimator provides the estimated noise level. 

15 [0017] An embodiment of the spatio-temporal filter unit 

according to the invention comprises a noise estimator designed to 
estimate a noise level in the current image and to control 
thresholds of the adaptive sigma filter based on the estimated 
noise level. The thresholds can be increased in case of a 

20 relatively high estimated noise level and decreased in case of a 
relatively low estimated noise level. Therefore, optimal noise 
filtering is achieved at all noise levels. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0018] These and other aspects of the spatio-temporal filter 
unit and the image display apparatus according to the invention 
will become apparent from and will be elucidated with reference 
5 with respect to the implementations and embodiments described 
hereinafter and with reference to the accompanying drawings, 
wherein: 

[0019] Fig. 1 schematically shows an embodiment of a spatio- 
temporal filter unit; 
10 [0020] Fig. 2 schematically shows an embodiment of a spatio- 
temporal filter unit comprising a noise estimator, a motion 
estimator and a motion detector; and 

[0021] Fig. 3 shows elements of an image display apparatus. 

15 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0022] Fig. 1 schematically shows an embodiment of a spatio- 
temporal filter unit 100 and two memory units which are used by the 
spatio-temporal filter unit 100. These memory units might be shared 
with other units in a system comprising the spatio-temporal filter 

20 unit 100. The first memory unit is a picture memory 114 for storing 
the output of the spatio-temporal filter unit, being a temporally 
recursive filtered image. The second memory unit is a cache 116. 
The signal representing the images enters the spatio-temporal 
filter unit 100 at the connection 102. The filtered signal leaves 

25 the spatio-temporal filter unit 100 at the connection 118. Pixels 
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from this filtered signal are also stored in the picture memory 
114. Pixels from an image that enter the spatio-temporal filter 
unit 100 are buffered in the spatial pixel buffer 104. The spatial 
pixel selector 106 selects pixels from the spatial pixel buffer 104 
5 and provides these selected pixels to the filter kernel 107 of the 
sigma filter 112. Pixels from the picture memory 114 are cached in 
the cache 116 for reasons of performance. From the cache 116 # 
pixels are buffered in the temporal pixel buffer 108. The temporal 
pixel selector 110 selects pixels from the temporal pixel buffer 
10 108 and also provides these selected pixels to the filter kernel 
107 of the sigma filter 112. In this way, the filter kernel 107 
operates on the pixels from both the spatial pixel selector 106 and 
the temporal pixel selector 110. 

[0023] Fig. 2 schematically shows a relatively more 
15 sophisticated embodiment of a spatio-temporal filter 200. The white 
boxes in Fig. 2 are part of the spatio-temporal filter 200. The 
shaded boxes in Fig. 2 are not part of the spatio-temporal filter 
200. These shaded boxes represent memory units. These might be 
shared with other units in a system comprising the spatio-temporal 
20 filter unit 200. The first memory unit is a picture memory 214 for 
storing the output of the spatio-temporal filter unit, being a 
temporally recursive filtered image. The second memory unit is a 
cache 216. The signal representing the images enters the spatio- 
temporal filter unit 200 at the connection 202. The filtered signal 
25 leaves the spatio-temporal filter unit 200 at the connection 218. 
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Pixels from this filtered signal are also stored in the picture 
memory 214. Pixels from an image that enters the spatio-temporal 
filter unit 200 are buffered in the spatial pixel buffer 204. The 
spatial pixel selector 206 selects pixels from the spatial pixel 
5 buffer 204 and provides these selected pixels to the filter kernel 
207 of the sigma filter 212. Pixels from the picture memory 214 are 
cached in the cache 216 for reasons of performance. From the cache 
216, pixels are buffered in the temporal pixel buffer 208. The 
temporal pixel selector 210 selects pixels from the temporal pixel 

10 buffer 208 and also provides these selected pixels to the filter 
kernel 207 of the sigma filter 212. In this way, the filter kernel 
207 operates on the pixels from both the spatial pixel selector 206 
and the temporal pixel selector 210. The spatio-temporal filter 200 
comprises a motion detector 224 designed to detect motion and to 

15 control the aperture of the temporal pixel selector 210 based on 

detected motion. The spatio-temporal filter 200 further comprises a 
motion estimator 222 designed to supply motion vectors and to 
control the position of the temporal pixel selector 210 related to 
the temporal pixel buffer 208, based on the motion vectors supplied 

20 by the motion estimator. The spatio-temporal filter 200 also 

comprises a noise estimator 220 for estimating a noise level in the 
current image and for controlling the aperture of at least one of 
the pixel selectors, being the temporal pixel selector 210 and the 
spatial pixel selector 206, based on the estimated noise level. In 
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addition, the noise estimator 220 controls thresholds of the 
adaptive sigma filter 212, based on the estimated noise level. 
[0024] Fig, 3 shows elements of an image display apparatus 300 
according to the invention. The image display apparatus 3 00 has a 
5 receiving means 3 02 for receiving a video signal representing the 
images to be displayed. The signal may be a broadcast signal 
received via an antenna or cable, but may also be a signal from a 
storage device, like a VCR (Video Cassette Recorder) or DVD 
(Digital Versatile Disk) . The image display apparatus 3 00 further 

10 has a spatio-temporal filter unit 304 for filtering the video 
signal and a display device 306 for displaying the images 
represented by the filtered video signal. The spatio-temporal 
filter unit 304 is implemented as described in Fig. 1 or Fig. 2. 
[0025] It should be noted that the above-mentioned embodiments 

15 illustrate rather than limit the invention and that those skilled 
in the art will be able to design alternative embodiments without 
departing from the scope of the appended claims. In the claims, any 
reference signs placed between parentheses shall not be constructed 
as limiting the claim. The word 'comprising' does not exclude the 

20 presence of elements or steps other than those listed in a claim. 
The word "a" or "an" preceding an element does not exclude the 
presence of a plurality of such elements. The invention can be 
implemented by means of hardware comprising several distinct 
elements and by means of a suitable programmed computer. In the 
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unit claims enumerating several means, several of these means can 
be embodied by one and the same item of hardware. 
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